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Abstract. An open question about Gabidulin codes is whether polynomial-time list 
. decoding beyond half the minimum distance is possible or not. In this contribution, 

we give a lower and an upper bound on the list size, i.e., the number of codewords 
in a ball around the received word. The lower bound shows that if the radius of this 
£\J ■ ball is greater than the Johnson radius, this list size can be exponential and hence, 

no polynomial-time list decoding is possible. The upper bound on the list size uses 
subspace properties. 

c/5 ■ 1 Introduction 

o m 

Gabidulin codes [1] can be seen as the analogs of Reed-Solomon (RS) codes 
in rank metric. There are several efficient decoding algorithms up to half 
the minimum rank distance. However, in contrast to RS codes, there is no 
polynomial-time decoding algorithm beyond half the minimum distance. For 
• RS codes, it can be shown that the number of codewords in a ball around any 

received word is always polynomial in the length when the radius of the ball 
is at most the Johnson radius. The Guruswami-Sudan algorithm provides an 
efficient polynomial-time list decoding algorithm of RS codes up to the Johnson 
CN ' radius. 

For Gabidulin codes, there is no polynomial-time list decoding algorithm; it 
is not even known, whether such an algorithm can exist or not. An exponential 
lower bound on the number of codewords in a ball of radius r around the re- 
ceived word would prohibit polynomial-time list decoding since the list size can 
be exponential, whereas a polynomial upper bound would show that it might 
be possible. Faure [2] and Augot and Loidreau [3] made first investigations of 
this problem. 

In this paper, we provide a lower and an upper bound on the list size. 
The lower bound shows that the list size can be exponential in the length 
when the radius is at least the Johnson radius and therefore in this region, no 
polynomial-time list decoding is possible. The upper bound uses the properties 
of subspaces and gives a good estimate of the number of codewords in such a 
ball, but is exponential in the length and therefore does not provide an answer 
to polynomial-time list decodability in the region up to the Johnson bound. 
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2 PRELIMINARIES 



2 Preliminaries 

2.1 Definitions and Notations 

Let q be a power of a prime, let F g denote the finite field of order q and let 

F g m be the extension field of degree m over ¥ q . We denote = x q% for any 
integer i, then a linearized polynomial, introduced by Ore jlj, over ¥ q m has the 

form f(x) = Y^iLo fi x ^\ with /j G F g m. If the coefficient fd f ^ 0, we call 

df = deg g /(x) the q-degree of /(x). For all a\,a2 € F g and V a, 6 6 F g m, the 
following holds: /(aia + a^fr) = ctif(fl) + oi2f(b). The (usual) addition and the 
non-commutative composition f{g{x)) (also called symbolic product) convert 
the set of linearized polynomials into a non-commutative ring with the identity 
element = x. In the following, all polynomials are linearized polynomials. 

Given a basis of ¥ q ™ over ¥ q , there exists a one-to-one mapping for each 
vector x G ¥ qm on a matrix X G F™ xn . Let rank(x) denote the (usual) rank of 
X over ¥ q and let 7£(x) = 7£(X) denote the row space of X over F g . The kernel 
of a matrix is denoted by ker(x) = ker(X) and the image by im(x) = im(X). 
For an m x n matrix, if dimker(x) = t, then dimim(x) = rank(x) = n — t. 
Throughout this paper, we use the notation as vector or matrix equivalently, 
whatever is more convenient. The minimum rank distance d of a code C is 
defined by 

d = min{rank(c) | c G C, c ^ 0}. 

A Gabidulin code can be defined by the evaluation of degree-restricted linearized 
polynomials as follows. 

Definition 1 (Gabidulin Code pQ). A linear Q(n,k) Gabidulin code of length 
n and dimension k over ¥ q m for n < m is the set of all codewords, which are 
the evaluation of a q-degree restricted linearized polynomial f(x): 

g(n,k) d = {c = (f(a ),f(a 1 ),.. . ,/(a n _i)| deg q f(x) < k)}, 
where the fixed elements ceo, ■ ■ ■ > c^n-l £ ¥ q ™ are linearly independent over ¥ q . 

Gabidulin codes are Maximum Rank Distance (MRD) codes, i.e., they fulfill 
the rank metric Singleton bound with equality and d = n — k + 1 pQ. 

The number of s-dimensional subspaces of an n-dimensional vector space 
over ¥ q is the Gaussian binomial, calculated by 

def tt q n -q l 



n 



0« 



with the upper and lower bounds g s ( n-s ) < [ ™ ] < 4g s ("~ s ) . 

Moreover, B T (&) denotes a ball of radius r in rank metric around a word 
a G F™ m . The volume of T (a) is independent of its center and is simply the 
number ofmxn matrices of rank less than or equal to r. 



2.2 Problem Statement 
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2.2 Problem Statement 

Problem 1 (Maximum List Size). Let the Gabidulin code Q(n, k) overF q m with 
n < m and d = n — k + 1 be given. Let r < d. Find a lower and upper bound 
on the maximum number of codewords £ in the ball of rank radius r around 
r = (ro r\ ... r n _i) € F" m . Hence, find a bound on 

£ A M max (|B r (r)ng|). 

For an upper bound, we have to show that the bound holds for any received 
word r, whereas for a lower bound it is sufficient to show that there exists one 
r for which this bound on the list size is valid. 

Let C = {ci, C2, • • • , eg} with \C\ = £ denote the list of all codewords in the 
ball of radius r around r, i.e., Cj € 0(n, k) and rank(r — Cj) < t, for i = 1, . . . , £. 



3 A Lower Bound on the List Size 

Faure showed with a probabilistic approach in [2] that the maximum list size 
of Gabidulin codes with n = m is exponential in n for r > n— \Jn(n — d). Our 
bound slightly improves this value and uses a different proving strategy, based 
on evaluating linearized polynomials. This approach is inspired by Justesen and 
Hoholdt's [5 J and Ben-Sasson, Kopparty, and Radhakrishna's [6] approaches for 
bounding the list size of RS codes. 

Theorem 1 (Lower Bound on the List Size). Let the Gabidulin code Q(n,k) 
over Fqm with n < m and d = n — k + 1 be given. Let r < d. Then, there exists 
a word r € F" m such that 

r n I 

£ > \BJr) ng\> . [ "~ TJ , = q m q r(m+n)-r 2 -md (1) 
— 1 ' \ / -'I — („m\n—T—k i ~i \ j 



and for the special case of n = m: £ > 



~.n Jlrir —t —nd 



Proof. Since t < d = n — k + 1, also k — 1 < n — r holds. Consider all monic 
linearized polynomials of (/-degree exactly n — t with a root space of dimension 
n — r, where all roots are in F g n. There are exactly (see e.g. Theorem 11.52]) 
[n-r] such polynomials. Now, let us consider a subset of these polynomials, 
denoted by V: all polynomials where the g-monomials of (/-degree greater than 
or equal to k have the same coefficients. Due to Dirichlet's principle there exist 
coefficients such that the number of such polynomials is 

vp\ > ["" T J 
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3 A LOWER BOUND ON THE LIST SIZE 



since there are ^ m ) n - T - fc possibilities to choose the highest n — r — (k — 1) 
coefficients of a monic linearized polynomial over ¥ q m. 

Note that the difference between any two polynomials in V is a linearized 
polynomial of g-degree strictly less than k and therefore the evaluation polyno- 
mial of a codeword of Q(n,k). Let r be the evaluation of f{x) € V at a basis 
A = {a ,ai,...,a n _i} ofF g n over ¥ q : 

r = (r n ... r n _i) = (/(a ) /(ai) . . . /(a n _i)). 

Further, let also G "P, then /(x) — 5(2;) has g-degree less than k. Let c 
denote the evaluation of f(x) — g(x) at A. Then, r — c is the evaluation of 
f(x) — f(x) + g(x) = g(x) 6 V, whose root space has dimension n — r and all 
roots are in F g n. Thus, dimker(r — c) = n — r and dimim(r — c) = rk(r — c) = t. 
Therefore, for any g(x) € V, the evaluation of f(x) — g(x) is a codeword from 
Q(n, k) and has rank distance r from r. This provides the following lower bound 
on the maximum list size: 

£ [ n—r J ^ H m r(m+n)—T —md 

— (g m ) n— T ~ ^ — ^q'm'jn—T—k " " ' 

and for n = m the special case follows. □ 

This lower bound is valid for any r < d, but we want to know, which is the 
smallest value for r such that this expression is exponential in n. For n = m, 
we can rewrite ([T]) by 



> 



n(l— e) _ — — nd-\-T 



where the first part is exponential in n for any < e < 1. The second exponent 
is positive for 

t >n — y/ n(n -d + e) d = t lb . (2) 

Therefore, our lower bound ([1]) shows that the maximum list size is exponential 
in n for r > tlb- For e = 0, the value tlb gives exactly the Johnson radius for 
Hamming metric. 

This reveals a difference between the known limits to list decoding of Ga- 
bidulin and RS codes. For RS codes, polynomial-time list decoding up to the 
Johnson radius is guaranteed by the Guruswami-Sudan algorithm. However, 
it is not proven that the Johnson radius is tight for RS codes, i.e., it is not 
known if the list size is polynomial between the Johnson radius and the known 
exponential lower bounds (see e.g. [5j|6]). For Gabidulin codes, we have shown 
that the maximum list size is exponential for r > tlb, which is asymptotically 
equal to the Hamming metric Johnson radius. 

Example 1. For the Gabidulin code Q(n = 12, k = 6) with d = 7, the Bounded 
Minimum Distance decoding radius is tbmd = \_[d — l)/2j = 3, the lower bound 
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by Faure (equivalent to the Hamming metric Johnson radius) is tj = [4.2] = 5 
and ([2D with e = 0.9 gives tlb = [3.58] = 4. This means for this code of rate 
k/n = 1/2, no polynomial time list-decoding beyond tbmd is possible. 



4 An Upper Bound on the List Size 

The following lemma shows that the row spaces of r — Cj and r — Cj, Cj, Cj £ £, 
i ^ j, have no (2r — d + l)-dimensional subspace in common. 

Lemma 1. Let r < d and let r E F^ m . Let Cj, /or z = 1, . . . , I, be codewords of 
the Gabidulin code G(n, k) with minimum rank distance d and let rk(r — Cj) < r 
hold for all i = 1, . . . , I. Let rk(r — Cj) = ti < r and rk(r — Cj) = tj < t, % ^ j. 
Then, the row spaces of (r — Cj) and (r — Cj) Ziaue no subspace of dimension at 
least ti + tj — d + 1 in common, for [(d — l)/2j < tj, tj < r. 

Proof. Assume, there exist (r — Cj) and (r — Cj), with rank(r — Cj) = tj, rank(r — 
Cj) = tj, i 7^ j, such that their row spaces contain the same subspace of 
dimension at least (ti + tj — d + 1). Then, 



r — c 
r — c 



dim(7£(cj — Cj)) = dim(7?.(r — Cj — r + Cj)) < dim ^TZ 
<U + tj - (U + tj - d+l) = d-l, 

which is a contradiction to rk(cj — Cj) > d. □ 

This means in particular, if rk(r — Cj) = rk(r — Cj) = t < r, they have no 
subspace of dimension at least 2t — d + 1 in common. Based on this lemma, we 
obtain the following upper bound on the maximum list size. 

Theorem 2 (Upper Bound on the List Size). Let the Gabidulin code Q(n,k) 
over ¥ q m with n < m and d = n — k + 1 be given. Let r < d. Then, for any 
word r G F" m and hence, for the maximum list size, the following holds 

1= max (\B T (j)r\Q\) < Y j»±i=4 (3) 



t=L(d-i)/2j+i [a+i-dJ 

r 

< 4 g (2t-d+l)(n-t) < 4 ( r _ L^fiJ) • g (2T-d+l)(r J1 -L(d-l)/2j -1) _ ^ 

t=L(<i-l)/2j+l 

Proof. We consider all words in F™m with n < m, therefore these words can be 
seen as matrices in an n-dimensional space. For any t, where [(d — l)/2j < t < 
d, there are [ 2 t-d+i ] subspaces of dimension 2t — d+ 1. Let r be any fixed word 
in Fgm and all codewords in C have rk(r — c») < r. Each r — Cj, for i = 1, . . . , £, 
of rank t < r contains [ 2t-d+l ] subspaces of dimension 2t — d + 1. 
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Due to LemmaHJ different r — Cj have no (2t — d+l)-dimensional subspace in 
common and therefore there are at most [24-^+1] / Nt+i-d] possible codewords 
in rank distance t to the word r. We sum this up for t from [(d — l)/2j + 1 up 
to r and we obtain (J3J) - 

With the bounds for the Gaussian binomial and since [(d — 1)/2J + 1 < t < 
r, the upper bound from @ follows. □ 

Note that for the special case r = d/2 and even minimum distance d, the 
upper bound from ((3|) is the bound from [H Equation (4)], which is 

n n—d/2 n n—d c .—t a~^ T a n 1 \ n ~\ 

y y / n, -t \ y y _y l 1 



£ < C « _ 1) 1 1 = ( a n _ 1) 

> q n_ 2q n-d/2 + q n-d W ^ J _ + g -2r g r _ X [r] • 

Thus, we have proved an upper bound on the maximum list size of a Gabi- 
dulin code. Unfortunately, this upper bound is exponential in n < m for any 
r > l(d — l)/2j and therefore does not provide any conclusion if polynomial- 
time list decoding is possible or not in the region up to the Johnson bound. 
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